10. Training a Profitable Agent

AI For Trading C5 L3 A04 Training A Profitable Agent V4

Developing a Profitable Training Agent

This summary outlines strategies for creating a successful training agent using Deep Q-Networks (DQN).

Key Considerations in Training Process:

  1. Training Parameters:

    • Episodes (E): A balance is required; too few may prevent convergence, too many may lead to overfitting.
    • Mini Batch Size (M): Smaller sizes introduce noise (potential regularization) but may hinder convergence; larger sizes reduce noise and quicken convergence but increase overfitting risks.

  2. Monitoring Training:

    • Track Loss: Monitor training and validation losses. An increasing validation loss with decreasing training loss indicates overfitting.
    • Trade Evaluation: Compare decreasing DQN loss against actual trading performance for profitability.
    • Model Checkpoints: Regularly save models to revert if performance degrades.

  3. Comparative Analysis:

    • Random Seed Setting: Allows consistent reproducibility across training rounds.
    • Test Dataset Run: Conduct a test episode to assess model generalization from training to new data.

  4. Adjustment & Validation:

    • Use validation to determine the optimal number of episodes, adjusting mini batch size as necessary for stable training.

What is the effect of having a very small number of training episodes (E) in the training process?
SOLUTION: The DQN will not have enough data to learn effectively and may not converge

What should you do if you observe that training loss is decreasing but profitability is decreasing as well?
SOLUTION: Reevaluate and possibly adjust reward functions, feature selections, or DQN architecture

Why is it important to save model checkpoints during training?
SOLUTION: To avoid starting training from scratch if a problem is detected later

Next Concept